Evolutionary games and the emergence of complex networks 
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The emergence of complex networks from evolutionary games is studied occurring when agents are allowed 
to switch interaction partners. For this purpose a coevolutionary iterated Prisoner's Dilemma game is defined 
on a random network with agents as nodes and games along the links. The agents change their neighborhoods 
to improve their payoff. The system relaxes to stationary states corresponding to cooperative Nash equilibria 
with the additional property that no agent can improve its payoff by changing its neighborhood. Small pertur- 
bations of the system lead to avalanches of strategy readjustments reestablishing equilibrium. As a result of the 
dynamics, the network of interactions develops non-trivial topological properties as a broad degree distribution 
suggesting scale-free behavior, small-world characteristics, and assortative mixing. 
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Economic relationships evolve: If the fish you bought on 
the fish market is spoiled you will probably switch the dealer 
next time. On larger scales, markets may develop into com- 
plex interaction structures solely from agents using their free- 
dom of choosing their partners. 

Statistical physics has a tradition of studying systems of in- 
teracting agents motivated by economic or social interactions. 
One example is evolutionary game theory on the lattice where 
agents can be thought of as particles with generalized interac- 
tions and internal memory. These properties give rise to com- 
plex spatiotemporal dynamics when agents try to locally opti- 
mize their payoffs [JjJ ^J, Bj, The goal of this paper is to relax 
a further constraint in this class of models, namely the fixed 
interaction topology. Agents may use the choice of interaction 
partners as another means to locally optimize their payoff and, 
thereby, shape the interaction matrix and the topology of the 
system. 

An interesting aspect of agent systems with evolving neigh- 
borhoods is the structure of the emerging interaction net- 
works. The theoretical physics community has focused on 
complex network structures in diverse natural systems re- 
cently [Q, ||]. Game theoretic models on evolving networks 
may provide a fresh view on the emergence of complex net- 
works in systems where strategic interactions occur. One ex- 
ample are economic systems where complex global structures 
and dynamics are observed [||, 0]. 

The dynamics of games on static networks has been stud- 
ied recently, namely the question of stability of cooperative 
behavior in the iterated Prisoner's Dilemma (IPD) H pi [To| ], 
also under the influence of noise in the spatial relationships 
of agents JTT|]. The question how the game itself may affect 
network topology was asked by Zimmermann et al. in a sim- 
ple setting of cooperating and defecting players, leading to 
a network that self-organizes into a steady state with highly 
connected cooperators [jl2j]. 

In the following, let us consider the IPD with memory on 
an evolving network. Details of this game on static networks 
can be found in Ref. 0]. The Pr isoner s Dilemma Q is 
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Action 
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TABLE I: Representation of an agent's strategy (0, defection; 1, co- 
operation). A player with the above strategy cooperates, no matter 
whether its opponent has cooperated or defected in the last move. 



a two-person game defined by the payoff matrix 
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with the entries aij being the payoffs of agent 1 playing 
strategy Sj against agent 2 using strategy Sj (si, cooperat- 
ing; S2, defecting). In general, the entries have to fulfill 
ai2 < a 2 2 < an < a 2 i and a i2 + 021 < 2an. 

In the IPD, a strategy is interpreted as a map of an agent's 
knowledge to an action. Here, knowledge is defined by the 
number of the m previous moves the agent can remember. 
We will confine the strategy space to strategies with m = 1, 
i.e., one agent knows only about the latest move of its oppo- 
nent. At the beginning of an encounter, there is no previous 
move. Therefore, the first action is also part of the strategy 
(Table |). It can be viewed as a lookup table where each his- 
tory is mapped to an action. In the case of m = 1, there 
are 2 2 +1 = 8 possible strategies. The strategies "always de- 
fect" (000) and "tit for tat" (Oil) are always Nash equilibria of 
this finite normal form game. Initially, a random network of a 
given average degree (i.e., number of a node's next neighbors) 
(k) is generated resulting in a Poissonian degree distribution. 
Strategies are assigned to the players at random. The coevo- 
lutionary dynamics consists of one part for the evolution of 
the strategies and another for network evolution. In each iter- 
ation cycle the following steps take place. Strategy evolution: 
(i) One agent i is chosen randomly and its strategy is mutated 
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FIG. 1: Distribution P(M) of avalanche size M in the critical 
regime. P(M) exhibits scale-free behavior with a scaling exponent 
of 7 = —1.35 ± 0.04. The inset shows the sub- and supercritical 
regimes (N = 500, (k) = 5, a 2 i = 3.5, 5.0, 5.1). 
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FIG. 2: Degree distributions P(k) for critical (solid line, 021 = 5.0) 
and subcritical (dotted line, 0,21 = 3.5) avalanche dynamics. The 
dashed plot shows the initial Poissonian degree distribution. (N = 
500, (k) = 20). 



to a strategy picked at random, (ii) The mutated agent plays 
against its neighbors and its payoff is compared to its payoff 
before the mutation. In case of a payoff increase the mutation 
is accepted and the payoffs of all neighbors are updated. This 
strategy update has been first used in Ref. jlCjl. Step (ii) cor- 
responds to the assumptions that changing the strategy may 
bear some costs (risk) for the player and that mutations occur 
on a time scale slower than the time scale of the game. Net- 
work evolution: (i) With probability a, one randomly chosen 
agent i is connected to a new neighbor taken at random, (ii) 
If the new connection leads to an increase in average payoff, 
player i will accept the new connection and remove the link 
to the neighbor it scores worst against. In case player i had no 
neighbor before, it accepts the connection, and a link between 
a random pair of players will be removed to keep the average 
degree (fc) constant [f20|l, Thereafter, all payoffs are updated. 
Note that the information used by an agent (its own payoff) is 
fully local. 

The iteration of these steps eventually leads to stationary 
states. The choice of the value of a that determines the speed 
of network evolution is not critical and convergence has been 
verified for values between a = 0.001 and 1. The station- 
ary states, where no further increase in payoff is possible for 
any agent, are cooperative and dominated by the strategy "tit 
for tat" (011). With regard to strategies, the stationary states 
correspond to game theoretic Nash equilibria [15|. Network 
evolution here does not interfere with the notion of Nash equi- 
libria. In a stationary state no player can improve its payoff, 
neither by changing its strategy nor by choosing a different 
neighborhood. In the remainder of this paper, such an equilib- 
rium will be called a network Nash equilibrium. 

One essential property of this spatially extended coevolu- 
tionary game is the stability of the network Nash equilib- 
ria. Exploring the IPD on static networks [ffoh, perturbations 
of the game's Nash equilibria have been observed to cause 
avalanches of M mutation events until stationary states are 
reestablished. Depending on the crucial parameter of the 



payoff matrix, the temptation to defect a 2 i, three different 
regimes of avalanche dynamics were found. First, for low 
temptations, the distribution of avalanche size P(M) is sub- 
critical with few mutation events necessary to reach again an 
equilibrium. In an intermediate range of 021, scale-free be- 
havior occurs as the distribution of avalanche size exhibits a 
power law with exponent 7 = —1.39 ± 0.10. For high values 
of 021, the game becomes supercritical with a probability for 
very large avalanches much higher compared to the critical 
regime. 

Let us similarly perturb the network Nash equilibrium by 
randomly changing the strategy of an arbitrary player plj]. 
The deviating strategy offers new opportunities for strategy 
changes to the neighboring players. The perturbation then can 
spread which leads to an avalanche of mutation events. Addi- 
tionally, payoff can be improved by changing one's neighbor- 
hood which results in a network evolution closely connected 
to the strategy evolution of the individual agents. 

One finds that the model with network dynamics considered 
here results in a similar avalanche dynamics as observed for 
the static network case. For temptations to defect 021 < 4.8 
and CI21 > 5.2, the subcritical and supercritical regimes 
emerge (inset in Fig. [[]). The critical regime exists in the 
same range 4.9 < 021 < 5.1 as for static networks with 
same mean degree (Fig. [l]). In addition, the scaling exponent 
7 = —1.35 ± 0.04 agrees with the static case. Surprisingly, 
however, evolution of network structure is profoundly affected 
by the avalanche dynamics. 

After an avalanche, network evolution comes to an end 
when a network Nash equilibrium is reached. The equilibrium 
network, which emerges in the course of subsequent perturba- 
tions and avalanches, strongly differs from the initial random 
graph. For practical purposes we here speak of an equilibrium 
network when the first three cumulants of the degree distribu- 
tion do not change over a period of time much larger than the 
time needed to reach these values. The first three cumulants 
are given by the mean degree (k) (being constant because of 
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TABLE II: Cumulants of the degree distribution P(k) (N = 500, 
errors are given by standard deviations). 
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TABLE III: Clustering coefficient Ca, average shortest path length £, 
and Pearson coefficient r of the evolved networks (iV = 500, errors 
are given by standard deviations). 



step (ii) of the network evolution process), the variance a 2 and 
the skewness A3 = ((k — (fc)) 3 }/er 3 . The stationary values of 
the cumulants for different mean degrees and the three distinct 
regimes of avalanche dynamics are given in Table |n|. Indepen- 
dent of mean degree and temptation to defect, one observes a 
strong broadening of the degree distribution, up to an increase 
of the variance a 2 by a factor of 20 compared to the initial ran- 
dom network with Poissonian degree distribution ((Tq). The 
skewness A3 is significantly higher than expected for the Pois- 
sonian distribution (A3) indicating a more asymmetric distri- 
bution with a longer tail to the right. The dependence of the 
degree distribution on the different regimes of avalanche dy- 
namics can be seen in Fig. || where the degree distributions for 
critical and subcritical avalanche dynamics are plotted. In the 
critical regime (and similarly in the supercritical regime), the 
broad degree distribution resembles a power law. In the range 
of (k) where critical dynamics are possible JTc| ] the cutoff of 
the degree distribution grows with increasing (k), whereas the 
degree distribution is independent of the number of agents N. 
Note that although the quantity selected for is the sum of the 
payoffs an agent receives, it does not select for high number of 
neighbors as the number of neighbors is kept constant for the 
agent who decides. However, the agent who receives the link 
appears to be more or less attractive depending on its strategy 
and this reflects in the emerging link distribution. 

Besides the degree distribution, important properties when 
exploring a complex network's topology are also clustering 
and average shortest path length. The clustering coefficient 
Ca is a measure of the probability that two nodes share a 



common neighbor [|l6| 

3 x (number of fully connected triples) 



C A - 



number of triples 



(2) 



Ca is the fraction of fully connected triples, with a triple be- 
ing a connected subgraph which contains exactly three nodes. 
These values are compared to the clustering coefficient of a 
Poissonian random network Co and the clustering coefficient 
C we derived for a random network with identical degree dis- 
tribution but randomly assigned links fll7| ] 



C' = 



1 



(k 2 ) 
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Comparison of the experimental clustering Ca with Co 
shows an enhanced clustering (Table |l|). Since Ca is repro- 
duced well by Eq. (||) the increase in clustering can be entirely 
traced to the change of the degree distribution. 

The average I of the shortest path lengths between two 
nodes is determined for the giant component of the network 
always containing more than 95% of all nodes. Similarly 
to the treatment of the clustering coefficient, shortest path 
lengths for Poissonian random networks (£q) and for random 
networks with identical degree distributions (£') are calculated 
[fT^]. The observed values of £ are slightly larger than the av- 
erage shortest paths in a Poissonian random network resulting 
in still very short paths between any two nodes in the network 
(Table |ll]|). The estimate £' does not yield results as good as 
C in the case of the clustering coefficient. Obviously, for the 
shortest path length £, the assumption of randomly assigned 
links JIr| |l7[ ] is not sufficient for an accurate estimate. 

To gain further insight into the structure of the evolved net- 
works, the Pearson coefficient r is calculated. It provides a 
measure of the degree correlation between neighboring nodes 

[fir 



(4) 



= -tT2 ^(q{h, v) - Q{n)Q{v) 



Q(n) is the distribution of the remaining degree [i (i.e., /1 = 
k — 1) and Q(fj,,v) the joint probability distribution that the 
two nodes at the ends of a randomly chosen link have the re- 
maining degrees fi and v. A positive Pearson coefficient in- 
dicates that nodes with a high degree prefer linking to other 
highly connected nodes (assortative mixing). For r < high 
degree nodes tend to link to poorly connected agents (disas- 
sortative mixing). No preference is given when such correla- 
tions are absent, r = 0. The evolved networks studied here 
are neutral in the subcritical avalanche regime, as expected 
for the initial Poissonian random network (Table Q). How- 
ever, for critical and supercritical behavior, assortative mix- 
ing is observed (comparable in size to the assortative mixing 
of real-world graphs as, for example, co-authorship networks 

To summarize we have studied a model for a coevolution- 
ary Prisoner's Dilemma game on an evolving network where 
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each agent is allowed to alter its neighborhood in order to en- 
hance its payoff, thereby shaping the topology of the network. 
One observes that the network, after an initial perturbation 
and a subsequent avalanche of rearrangements, relaxes into a 
stationary state which is stable w.r.t. strategy variations (Nash 
equilibrium), as well as against local topology changes (net- 
work Nash equilibrium). In this equilibrium, even a change of 
topology cannot enhance any player's payoff. While the dy- 
namics of the relaxation process is similar to the correspond- 
ing model on static networks without topology updates, the 
equilibrium structure of the network, emerged in the course of 
avalanches, strongly deviates from the initial random graph. 
The network evolves to a statistically stationary state with a 
broad degree distribution, suggesting scale-free behavior in 
the critical avalanche regime. It exhibits small-world behavior 
[Jjljl, and in the critical and supercritical regimes also shows 
assortative mixing, a property of social networks not shared 
by many standard models [ |l8| ] . 

The spatial coevolutionary game studied here is a minimal 
non-trivial model of network evolution driven by game theo- 
retic interactions. It may help in understanding the emergence 
of global organization from local interactions as observed in 
social and economic systems. It also provides an alternative 
mechanism for the evolution of complex networks possibly 
relevant for real-world systems where simple growth models 
as, for example, preferential linking are not easily appli- 
cable. 
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